Optimal Information Collection Policies in a Markov Decision Process Framework

Constrained Markov Decision Process and Optimal Policies

In the course lectures, we have discussed a lot regarding unconstrained Markov Decision Process (MDP). The dynamic programming decomposition and optimal policies with MDP are also given. However, in this report we are going to discuss a different MDP model, which is constrained MDP. There are many realistic demand of studying constrained MDP. For instance, in the wireless sensors networks, each...

Identification of optimal policies in Markov decision processes

In this note we focus attention on identifying optimal policies and on elimination suboptimal policies minimizing optimality criteria in discrete-time Markov decision processes with finite state space and compact action set. We present unified approach to value iteration algorithms that enables to generate lower and upper bounds on optimal values, as well as on the current policy. Using the mod...

Optimal Maintenance and Improvement Policies in Water Distribution System: Markov Decision Process Approach

The Markov decision process (MDP) based methodology is implemented in order to establish the optimal schedule which minimizes the cost. Formulation of MDP problem is presented using the information about the current state of pipe, improvement cost, failure cost and pipe deterioration model. The objective function and detailed algorithm of dynamic programming (DP) are modified due to the difficu...

Discovery of Structured Optimal Policies in Markov Decision Processes

In this chapter we continue work on vfd, the novel method for discovery of relative value functions for Markov Decision Processes that we introduced in Chapter 6. vfd discovers algebraic descriptions of relative value functions using ideas from the Evolutionary Algorithm eld and, in particular, these descriptions include the model parameters of the mdp. We extend that work and demonstrate how a...

Optimal Policies for Partially Observable Markov Decision Processes